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Abstract 


While  variants  of  the  steepest  edge  pivoting  rule  are  commonly  used  in  linear  programming 
codes  they  are  not  known  to  have  the  theoretically  attractive  property  of  avoiding  an  in¬ 
finite  sequence  of  pivots  at  points  of  degeneracy.  A  natural  extension  of  the  steepest  edge 
pivoting  rule  based  on  steepest  ascent  is  developed  and  shown  to  be  provably  finite.  An 
alternative  finite  pivoting  procedure  that  is  computationally  more  attractive  than  steepest 
ascent  is  then  introduced  and  it  is  argued  that  with  probability  1  the  procedure  has  the 
same  computational  requirements  as  steepest  edge  independent  of  the  linear  program  being 
solved.  Both  procedures  have  the  unique  advantage  that  they  choose  the  pivot  element 
without  explicit  knowledge  of  the  set  of  all  active  constraints  at  a  point  of  degeneracy, 
thus  making  them  attractive  in  combinatorial  settings  where  the  linear  program  is  never 


explicitly  written  out. 


1  Introduction 


Degeneracy  in  simplex  algorithms  for  solving  linear  programs  has  long  been  recognized  as 
both  a  theoretical  and  practical  problem.  From  a  theoretical  perspective  the  problem  was 
resolved  early  in  the  history  of  linear  programming.  The  perturbation/lexi cographic  method 
(Charnes,  1953;  Dantzig,  Orden,  and  Wolfe,  1955;  Wolfe,  1963)  avoids  cycling  by  dictating 
the  leaving  variable  while  allowing  complete  freedom  in  the  choice  of  entering  variable 
(among  variables  with  a  reduced  cost  of  correct  sign).  Bland’s  combinatorial  rule  (1977) 
dictates  the  choice  of  both  entering  and  leaving  variable.  More  recently,  Magnanti  and 
Orlin  (1988)  proposed  a  rule  for  the  parametric  simplex  algorithm  that  avoids  degeneracy 
using  a  lexicographic  method  that  dictates  the  entering  variable  while  maintaining  complete 
freedom  in  the  choice  of  leaving  variable  (among  variables  that  would  drive  the  solution 
infeasible). 

The  practical  difficulties  associated  with  degeneracy  have  come  to  be  fully  appreciated 
far  more  slowly.  To  this  day  many  texts  on  linear  programming,  having  resolved  the  problem 
of  degeneracy  theoretically,  routinely  pronounce  that  degeneracy  is  not  a  practical  problem 
based  on  the  observation  that  cycling  rarely  occurs.  Yet,  even  if  cycling  does  not  occur,  it  is 
nonetheless  possible  to  encounter  a  very  long  sequence  of  degenerate  pivots.  Cunningham 
(1979)  discussed  the  possibility  of  encountering  an  exponential  number  of  such  pivots  with¬ 
out  cycling  in  the  context  of  the  network  simplex  algorithm  and  introduced  the  term  stalling 
for  this  phenomenon.  While  all  linear  programs  arising  in  practice  are  degenerate  to  some 
degree,  the  number  of  degenerate  pivots  encountered  by  a  simplex  algorithm  is  profoundly 
influenced  by  the  pivot  selection  rule.  Ryan  and  Osborne  (1988)  reported  that  due  to  de¬ 
generacy  they  were  unable  to  solve  an  airline  crew  scheduling  problem  after  approximately 
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2000  iterations  when  using  a  pivot  rule  based  solely  on  reduced  cost  information,  but  that 
they  solved  the  problem  in  436  iterations  using  Wolfe’s  rule  (1963).  The  author  (Boyd, 
1990)  was  able  to  solve  some  extremely  degenerate  combinatorial  linear  programs  only  af¬ 
ter  employing  the  steepest  edge  pivot  rule.  For  linear  programs  demonstrating  significant 
degeneracy  —  such  as  linear  programs  arising  in  combinatorial  contexts  —  the  choice  of 
pivot  rule  can  mean  the  difference  between  solving  or  not  solving  a  linear  program. 

One  pivot  rule  of  particular  importance  is  steepest  edge.  The  steepest  edge  pivot  rule 
consists  of  choosing  the  edge  of  ascent  relative  to  a  given  basis  that  gives  the  greatest  change 
in  the  objective  function  per  unit  of  movement  along  that  edge.  Steepest  edge  and  its  vari¬ 
ants  have  been  known  and  used  for  decades  in  the  solution  of  large-scale  linear  programs 
and  many  papers  have  been  written  on  the  subject,  including  Crowder  and  Hattingh  (1974), 
Dickson  and  Frederick  (1960),  Goldfarb  and  Reid  (1977),  Harris  (1973),  Kuhn  and  Quandt 
(1963),  and  Wolfe  and  Cutler  (1963).  The  major  conclusion  reached  by  these  studies  is  that 
steepest  edge  often  dramatically  reduces  the  number  of  pivots  required  by  the  simplex  algo¬ 
rithm.  Recent  extensive  computational  results  supporting  this  claim  have  been  obtained  by 
Bixby  (1991)  and  by  Forrest  and  Goldfarb  (1991).  In  spite  of  the  additional  computational 
work  required  at  each  iteration  over  simpler  pivot  rules,  steepest  edge  often  proves  to  be  the 
pivot  rule  of  choice  due  to  the  increased  speed  attained  by  a  smaller  number  of  iterations. 
Hoffman  and  Padberg  (1991)  have  reported  marked  improvement  in  the  solution  times  of 
airline  crew  scheduling  problems  with  the  use  of  steepest  edge.  Commercial  codes  such  as 
IBM’s  MPSX  and  OSL  both  have  options  for  DEVEX  pricing  (an  approximate  steepest 
edge  procedure)  and  CPLEX  makes  use  of  primal  and  dual  steepest  edge  pricing. 

Beyond  the  overall  effectiveness  of  steepest  edge,  a  common  belief  within  the  linear 
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programming  community  is  that  steepest  edge  generates  relatively  few  pivots  at  points  of 
degeneracy.  The  author  became  interested  in  the  steepest  edge  pivot  rule  when  it  was 
suggested  as  a  way  to  overcome  degeneracy  encountered  in  his  own  applications.  However, 
while  steepest  edge  works  well  in  practice  it  remains  an  open  question  as  to  whether  or 
not  steepest  edge  is  a  true  degeneracy  resolution  procedure;  that  is,  a  procedure  that  is 
guaranteed  to  determine  an  edge  of  ascent  or  a  proof  that  no  such  edge  exists  after  a  finite 
number  of  pivots. 

In  this  paper  we  propose  two  simplex  pivot  rules  closely  related  to  steepest  edge  that  are 
degeneracy  resolution  procedures.  The  first  is  a  very  natural  extension  of  the  steepest  edge 
rule  based  on  steepest  ascent.  The  second  procedure  is  computationally  far  more  attractive 
than  steepest  ascent  and  it  is  argued  that  with  probability  1  the  procedure  has  the  same 
computational  requirements  as  steepest  edge  independent  of  the  linear  program  being  solved. 

The  procedures  are  different  than  all  previous  degeneracy  resolution  procedures  in  that 
they  rely  neither  on  lexicography  nor  ordering  arguments  such  as  those  found  in  Bland’s 
rule.  They  share  some  simlilarity  with  the  procedure  of  Magnanti  and  Orlin  in  that  there 
are  no  restrictions  placed  on  the  leaving  variable,  and  the  steepest  ascent  procedure  actu¬ 
ally  provides  some  freedom  in  the  choice  of  entering  variable.  However,  the  most  important 
theoretical  advantage  unique  to  the  procedures  is  that  the  pivot  element  is  chosen  without 
explicit  knowledge  of  the  set  of  all  active  constraints  at  a  point  of  degeneracy.  This  ad¬ 
vantage  is  particularly  useful  in  combinatorial  settings  where  the  linear  program  is  never 
explicitly  written  out. 
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2  Background 


In  order  to  make  the  combinatorial  implications  of  the  proposed  degeneracy  resolution 
procedures  explicit,  we  consider  a  linear  program  of  the  form 

max  cx 

(P)  s.t.  Ax  <  b 

where  A  is  to  X  n  and  is  assumed  to  have  rank  n  (the  problem  is,  of  course,  trivial  to  solve 
if  the  rank  is  less  than  n).  It  is  important  to  emphasize  that  while  the  form  (P)  will  be 
used  for  expository  purposes  the  procedures  to  be  described  can  be  applied  just  as  easily 
to  problems  in  a  more  standard  form  for  simplex  algorithms,  i.e.,  equality  constraints  and 
bounded  variables.  In  the  context  of  (P),  a  primal  simplex  algorithm  operates  as  follows. 

Algorithm  Simplex 

Given:  A  linearly  independent  collection  a1,.  ..,an  of  rows  of  A  such  that  the  intersection 
x  of  the  corresponding  constraints  is  feasible  for  (P). 

Purpose:  Solve  (P). 

1.  For  each  of  the  n  sets  of  points  Sk  =  {a1,..., a"}  -  {afc}  let  7 ka  <  0  be  the  unique 

direction  satisfying  7 kal  =  0  for  all  a 1  G  Sk  and  7 kak  <  0.  Let  d  =  7s  where  7s 
satisfies  c^s  >  0.  If  no  such  direction  exists,  stop;  the  present  solution  x  is  optimal. 

2.  Find  the  largest  6  such  that  A(x  +  §d)  <  b  and  a  row  r  of  A  such  that  ar(x  +  6d)  >  br 

for  9  >  0.  If  9  =  00,  stop;  (P)  is  unbounded. 

3.  Let  the  new  set  of  constraints  consist  of  {a1, . . . ,  a"}  U  { ar }  —  {a5}  and  return  to  step  1. 
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In  the  standard  form  for  simplex  algorithms  the  leaving  constraint  as  would  correspond 
to  an  entering  variable  and  the  entering  constraint  ar  would  correspond  to  a  leaving  variable. 
An  important  point  can  be  made  with  respect  to  step  2  of  algorithm  Simplex.  When  the 
constraints  Ax  <  b  are  explicitly  written  out  step  2  can  be  carried  out  enumeratively;  that 
is,  for  each  constraint  alx  <  6,-  the  value  9{  can  be  determined  such  that  al(x  +  9  id)  = 
bi  and  9,  ar ,  and  br  determined  by  the  minimum  non-negative  value  of  9t.  In  standard 
implementations  of  the  simplex  algorithm  this  is  what  actually  occurs.  However,  to  actually 
apply  algorithm  Simplex  all  that  is  necessary  is  an  oracle  for  solving  the  following  problem. 

Problem  STEP:  Given  a  point  x  satisfying  Ax  <  b  and  a  direction  d,  find  the  largest  6 
such  that  A(x  +  9d)  <  b  and  a  row  r  of  A  such  that  ar(x  -f  9d)  >  br  for  9  >  9. 

For  linear  programs  (P)  arising  in  combinatorial  contexts  it  is  often  the  case  that  m  »  n 
so  that  explicitly  writing  the  set  of  constraints  is  neither  desirable  nor  practically  feasible. 
However,  in  many  such  instances  an  oracle  for  solving  problem  STEP  does  exist.  Motiva¬ 
tion  for  the  present  work  stemmed  from  degeneracy  encountered  in  the  solution  of  knapsack 
separation  problems  (Boyd,  1990)  where  the  linear  programs  that  arise  have  an  exponen¬ 
tial  number  of  constraints  but  an  efficient  oracle  for  problem  STEP.  Algorithms  for  finding 
violated  subtour  elimination  constraints  of  the  traveling  salesman  polytope  can  be  modi¬ 
fied  to  serve  as  oracles  for  problem  STEP,  allowing  the  simplex  algorithm  to  be  used  to 
optimize  over  the  subtour  polytope  as  defined  in  Boyd  and  Pulleyblank  (1990)  in  spite  of 
an  exponential  number  of  constraints.  The  problem  of  finding  a  direction  of  ascent  on  the 
cone  defined  by  simple  cycle  inequalities  arises  in  the  solution  of  the  maximum  weight  cut 
problem  (Barahona  and  Titan,  1990;  Bixby  and  Saigal,  1991),  and  again  while  there  are  an 
exponential  number  of  such  constraints  there  exists  an  efficient  oracle  for  problem  STEP. 
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Yet,  while  oracles  often  exist  for  problem  STEP,  most  degeneracy  resolution  procedures 
must  place  requirements  on  the  entering  constraint  (exiting  variable  in  standard  simplex)  in 
order  to  guarantee  resolution  of  degeneracy.  A  notable  exception  is  the  parametric  algorithm 
of  Magnanti  and  Orlin  (1988),  although  as  a  parametric  algorithm  it  requires  more  than  an 
oracle  for  problem  STEP  in  order  to  be  implemented. 

In  addition,  all  degeneracy  resolution  procedures  proposed  to  date  require  global  infor¬ 
mation  on  the  set  of  active  constraints  at  a  point  of  degeneracy  in  order  to  make  a  pivot 
selection.  To  elaborate,  algorithm  Simplex  operates  by  successively  choosing  not  only  a 
feasible  point  x  but  a  set  of  n  constraints  with  linearly  independent  gradients.  In  the  con¬ 
text  of  degenerate  pivots,  it  is  the  choice  of  constraints  that  is  particularly  relevant  since  by 
definition  the  feasible  point  remains  unchanged.  Bland’s  rule  and  rules  that  use  lexicogra¬ 
phy  require  information  on  more  than  the  chosen  set  of  active  constraints  to  make  a  pivot 
selection.  However,  the  procedures  described  here  require  only  knowledge  of  the  chosen  set 
of  n  constraints  at  any  given  iteration  to  choose  a  pivot. 

Throughout  this  paper  we  let  ||  ■  ||  denote  the  euclidean  norm.  We  use  cone(a1, . .  .,an) 
to  denote  { a  €  IRn  :  a  =  uq a1  -f  . . .  +  unan,  c <q, . . .,  UJn  >  0}. 

3  Steepest  Edge 

The  steepest  edge  pivot  rule  defines  the  way  in  which  the  direction  d  is  chosen  in  step  1  of 
algorithm  Simplex.  Specifically,  d  is  chosen  as  the  direction  7s  with  the  largest  increase  in 
objective  function  value  per  unit  of  movement  along  7s;  that  is,  the  steepest  edge  of  the 
polyhedron  defined  by  the  chosen  collection  a1,..., a"  of  active  constraints.  We  formally 


6 


define  the  algorithm  in  the  context  of  (P)  for  comparison  with  algorithms  presented  in  the 
following  sections. 

Algorithm  Steep 

Given:  A  linearly  independent  collection  a1, . . . ,  an  of  rows  of  A  such  that  the  intersection 
x  of  the  corresponding  constraints  is  feasible  for  (P). 

Purpose:  Determine  a  direction  of  ascent  at  x  or  prove  that  no  such  direction  exists. 

1.  For  each  of  the  n  sets  of  points  Sk  =  {a1,..., a”}  —  {afc}  let  ~fka  <  0  be  the  unique 

direction  satisfying  7 kat  =  0  for  all  a 1  6  Sfc  and  7 kak  <  0. 

2.  Let  ■y3  maximize  v  =  (7*c)2/||7l||2  for  all  71,. .  .,7".  If  v  <  0,  stop;  no  ascent  direction 

exists. 

3.  Let  d  —  7s. 

4.  Let  6  and  aT  be  the  values  returned  by  the  oracle  for  problem  STEP  with  d  as  defined 

in  step  3.  If  6  0,  stop;  d  is  an  ascent  direction. 

5.  Replace  a 3  with  ar  and  return  to  step  1. 

4  Steepest  Ascent 

While  the  status  of  steepest  edge  as  a  theoretical  procedure  for  resolving  degeneracy  remains 
unresolved,  it  turns  out  that  a  close  variant  of  this  algorithm  based  on  steepest  ascent  is 
provably  a  degeneracy  resolution  procedure.  Specifically,  if  the  direction  d  sent  to  the  oracle 
for  solving  problem  STEP  is  a  direction  of  steepest  ascent  relative  to  the  active  constraints 
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(as  opposed  to  a  steepest  edge)  then  algorithm  Simplex  cannot  cycle.  Formally,  we  introduce 
the  following  algorithm  which,  given  a  vertex  of  the  feasible  region  of  (P),  finds  a  direction 
of  ascent  or  proves  that  no  such  direction  exists  after  a  finite  number  of  iterations. 

Algorithm  Ascent 

Given:  A  linearly  independent  collection  a1, . . . ,  an  of  rows  of  A  such  that  the  intersection 
x  of  the  corresponding  constraints  is  feasible  for  (P). 

Purpose:  Determine  a  direction  of  ascent  at  x  or  prove  that  no  such  direction  exists. 

1.  Find  a0  such  that  ||c  —  a°||  <  ||c  —  a||  for  all  a  £  cone(a1, . . . ,  an )  and  let  oq, . . .  ,u>n  >  0 

be  the  unique  values  such  that  a0  =  uq a1  +  •  •  •  +  ojnan.  If  a0  —  c,  stop;  no  ascent 
direction  exists.  Otherwise,  let  Iq  be  the  nonempty  set  of  indices  for  which  uq  =  0. 

2.  Let  6  and  ar  be  the  values  returned  by  the  oracle  for  problem  STEP  with  d  =  c  —  a0 .  If 

0  0,  stop;  c  —  a0  is  an  ascent  direction. 

3.  Let  as  be  such  that  s  £  I0  and  {a1, . . . ,  a"}U{ar}  —  {as}  is  linearly  independent.  Replace 

a*  with  ar  and  return  to  step  1. 

Algorithm  Ascent  is  a  steepest  ascent  algorithm  because  c—a°  is  the  direction  of  steepest 
ascent  relative  to  c  and  the  active  constraint  set  defined  by  al,...,an.  This  follows  from 
strong  duality  results  on  separation  found  in  the  convexity  theory  literature.  The  following 
proofs  actually  require  only  the  weaker  results  captured  in  the  following  lemma. 

Lemma  1  Let  a1 , . . . ,  an  £  IR"  be  a  collection  of  linearly  independent  points,  let  c  £  1R” 
be  such  that  c  £  cone(a1 , . . .  ,an),  and  let  a0  be  such  that  || c  -  o°||  <  || c  -  o||  for  all 
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a  G  cone(a1 , . . . ,  an) .  Finally,  let  uq  ,  >  0  be  the  unique  values  such  that  a0  = 

uqa1  +  •  •  •  +  u>nan  with  Iq  the  set  of  indices  for  which  =  0  and  iff  the  complement  of  Iq. 
Then 

1.  (c  —  a°)a  <  0  for  all  a  G  cone(a1, . . . ,  an), 

2.  (c  —  a°)a 8  =  0  for  all  a*  with  i  G  iff ,  and 

3.  (c  —  a°)c  >  0. 

Geometrically,  Lemma  1  is  obvious  and  we  omit  the  proof  as  it  is  easily  established  and 
one  of  many  standard  results  in  convexity  theory. 

Theorem  1  Algorithm  Ascent  terminates  after  a  finite  number  of  iterations  with  a  direc¬ 
tion  of  ascent  for  (P)  or  a  proof  that  no  such  direction  exists. 

Proof  .  We  begin  by  showing  that  when  the  algorithm  terminates  in  step  1  no  ascent 
direction  exists  and  when  it  terminates  in  step  2  an  ascent  direction  has  been  found.  The 
constraints  a1, . . . ,  an  are  initially  linearly  independent  and  the  algorithm  proceeds  beyond 
step  2  only  if  the  value  6  returned  by  the  oracle  for  solving  problem  STEP  is  0.  By  the 
definition  of  problem  STEP  this  means  the  constraint  ar x  <  br  returned  by  the  oracle 
satisfies  arx  =  bT ,  and  so  throughout  the  algorithm  all  constraints  a'x  <  6,  that  are  en¬ 
countered  are  satisfied  at  equality  by  x.  If  the  algorithm  terminates  in  step  1  with  c  —  a0 
then  c  =  uqa1  +  •  •  •  T  unan  with  u>i  >  0  and  thus  by  Farkas’  lemma  no  ascent  direction  in 
(P)  exists  at  x.  If  c  ^  a0  then  c  —  a0  is  clearly  a  direction  of  increasing  objective  function 
value  by  Lemma  1,  and  if  the  value  9  returned  by  the  oracle  for  solving  problem  STEP  is 
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nonzero,  as  is  the  case  when  the  algorithm  terminates  in  step  2,  then  by  the  definition  of 
problem  STEP  c  —  a0  is  a  feasible  direction  for  (P)  at  x. 

We  next  show  that  in  step  1  if  c  ^  a0  then  Iq  ^  0.  If  To  =  0  then  a0  resides  in  the 
strict  interior  of  cone(a1, . . . ,  an).  By  the  operation  of  the  algorithm  c  ^  a0  and  so  the  line 
segment  connecting  a0  and  c  contains  points  other  than  c,  and  since  ||  •  ||  is  a  norm  any  point 
a  a0  on  this  line  segment  satisfies  ||c  —  a||  <  ||c  —  a°||.  However,  since  a0  is  contained  in 
the  strict  interior  of  cone(a1, . . .  ,an )  some  point  a  /  a0  on  this  line  segment  also  must  be 
contained  in  cone(a1, . . . ,  an),  contradicting  the  definition  of  a0. 

To  see  that  a  vector  as  G  Iq  with  the  properties  assumed  in  step  3  exists,  note  that 
the  algorithm  only  proceeds  to  step  3  if  9  =  0,  and  by  the  definition  of  problem  STEP  it 
follows  that  aT  satisfies  (c  -  a°)ar  >  0.  Since  (c  -  a°)ar  ^  0  the  set  of  vectors  {a'  :  i  G  Iq}, 
which  satisfy  (c  —  a°)a'  =  0  by  Lemma  1,  together  with  aT  are  linearly  independent.  As  the 
vectors  a1, . . . ,  a"  are  linearly  independent  it  is  thus  possible  to  find  a  vector  as  with  s  €  Iq 
such  that  {a1, . . . ,  a"}  U  {(P}  —  {a3}  is  linearly  independent. 

We  conclude  by  demonstrating  that  the  algorithm  is  finite  by  showing  that  ||c  -  a°|| 
strictly  decreases  in  each  successive  iteration  of  the  algorithm.  Since  each  possible  set  of 
constraints  a1,..., a”  defines  a  unique  point  a0  (a0  represents  the  minimum  of  a  strictly 
convex  function  on  a  cone)  no  set  a1 , . . . ,  an  can  ever  be  repeated  and  finite  termination  of 
the  algorithm  follows.  Let  a1, . .  ,,an  be  the  new  set  of  vectors  constructed  in  step  3  of  the 
algorithm.  The  vector  aT  6  cone(a 1,  ...,an)  since  aT  =  ak  for  some  k  by  choice,  and  clearly 
a0  G  coneia1,. . .  ,a")  since  a0  can  be  written  as  a  non-negative  linear  combination  of  the 
vectors  {a*  :  i  G  Iq}  and  {a!  :  i  G  I§}  C  {a1, . . .,  a”}.  The  square  of  the  distance  between 
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c  and  points  on  the  line  segment  a0  +  0(ar  —  a0),  0  €  [0, 1],  connecting  a0  and  aT  is 
||c  -  (a0  +  0(ar  -  a0»||2 

=  ((c  -  a0)  -  0(ar  -  a°))((c  -  a0)  -  9(ar  -  a0)) 

=  (c  -  o°)(c  -  a0)  -  2 9(c  -  a°)(ar  -  a0)  +  02(ar  -  a°)(ar  -  a0) 

=  1 1 c  -  a°||2  -  20[(c  -  a°)ar  -  (c  -  a>°]  +  02\\ ar  -  a°||2. 

Since  (c  —  a°)ar  >  0  and  (c  —  a°)a°  =  0  it  follows  that  ||c  —  (a0  +  9(ar  —  <x° ) ) 1 1  <  ||c  —  a°||  for 
9  sufficiently  small  but  positive.  This  in  turn  implies  that  the  point  a?  satisfying  ||c  — o°||  < 

1 1  c  —  a||  for  all  a0  g  coneia1, ...  ,an)  satisfies  ||c  — a°||  <  ||c  —  a0 1|,  completing  the  proof.  □ 

It  is  possible  that  algorithm  Ascent  will  terminate  in  step  2  with  a  direction  of  ascent 
that  is  not  an  edge.  Practically  this  poses  no  difficulty  as  an  extreme  point  of  (P)  with  a 
better  objective  function  value  than  x  can  be  trivially  constructed  given  an  ascent  direction 
at  x,  and  finding  an  edge  of  ascent  at  x  is  not  significantly  more  difficult. 

The  choice  of  the  constraint  ar  returned  by  the  oracle  for  solving  problem  STEP  is 
irrelevant  in  guaranteeing  the  finiteness  of  algorithm  Ascent.  However,  it  will  have  an 
important  effect  on  the  speed  with  which  degeneracy  is  resolved  in  practice.  In  particular, 
if  an  ar  is  generated  that  greatly  decreases  the  distance  from  c  to  the  associated  cone  at 
each  iteration  then  rapid  resolution  of  degeneracy  should  follow. 

While  Theorem  1  does  not  prove  that  steepest  edge  is  a  degeneracy  resolution  procedure 
it  does  shed  some  light  on  why  steepest  edge  is  an  effective  degeneracy  resolution  procedure 
in  practice.  Often  the  closest  point  to  c  in  cone(a1, . . . ,  an)  may  well  be  a  facet  of  this  cone 
or,  equivalently,  the  direction  of  steepest  ascent  may  well  be  the  steepest  edge.  Even  if  the 
direction  of  steepest  ascent  is  not  the  steepest  edge,  the  steepest  edge  may  be  a  sufficiently 
good  approximation  to  the  direction  of  steepest  ascent  so  that  ||c-a°||  decreases  nonetheless. 
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5  Closest  Ascent 


While  steepest  ascent  is  theoretically  sound  it  relies  upon  finding  a  point  a0  minimizing 
||c  —  a0 1|  on  an  n- dimensional  cone  with  n  extreme  rays.  While  this  problem  can  be  solved 
finitely  using  an  active  set  strategy  it  nonetheless  remains  a  non-trivial  subproblem  that 
should  be  avoided  if  at  all  possible. 

In  this  section  we  present  a  degeneracy  resolution  procedure  that  depends  only  upon  the 
existence  of  an  oracle  for  solving  problem  STEP  but  finds  an  a0  minimizing  ||c  —  a°||  only 
as  a  last  resort.  In  fact,  following  the  discussion  of  the  algorithm  it  will  be  argued  that  the 
probability  of  having  to  solve  such  a  subproblem  is  0  independent  of  the  given  degenerate 
point. 


Algorithm  Close 

Given:  A  linearly  independent  collection  a1,. .  .,an  of  rows  of  A  such  that  the  intersection 
x  of  the  corresponding  constraints  is  feasible  for  (P). 

Purpose:  Determine  a  direction  of  ascent  at  x  or  prove  that  no  such  direction  exists. 

0.  Let  a0  be  any  point  in  the  interior  of  cone(a1, . . . ,  a"). 

1.  For  each  of  the  n  sets  of  points  Sk  —  {a1,..., a"}  —  { ak }  let  7 ka  <  0  be  the  unique 

constraint  satisfying  7 kal  =  0  for  all  a'  E  Sk  and  7 kak  <  0. 

2.  Let  a  >  0  be  the  maximum  value  of  a  such  that  7^(0°  +  a(c  —  a0))  <  0  for  all  71, . . . ,  7” 

and  let  K  be  the  set  of  indices  k  such  that  7 k(a°  +  o(c  —  a0))  =  0. 
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3.  if  | if  |  =  1 

Let  io  =  K ,  let  d  =  ~jk  where  k  €  K,  and  let  a0  =  a0  +  d(c  —  a0).  If  a  >  1,  stop;  no 
ascent  direction  exists. 

else 

Let  a0  be  such  that  ||c— a°||  <  ||c— a||  for  all  a  6  cone(a1, . . . ,  an)  and  let  u>i , . . . , uin  >  0 
be  the  unique  values  such  that  a0  =  uqa1  +  •  •  •  +  u>nan.  If  a0  =  c,  stop;  no  ascent 
direction  exists.  Otherwise,  let  Iq  be  the  nonempty  set  of  indices  for  which  —  0 
and  let  d  =  c  —  a0. 

4.  Let  6  and  ar  be  the  values  returned  by  the  oracle  for  problem  STEP  with  d  as  defined 

in  step  3.  If  6  ^  0,  stop;  d  is  an  ascent  direction. 

5.  Let  as  be  such  that  s  €  Io  and  {a1, . . .,  an}u{ar}  -  {as}  is  linearly  independent.  Replace 

a 3  with  ar  and  return  to  step  1. 

The  key  difference  between  algorithm  Ascent  and  algorithm  Close  lies  in  how  a0  is 
chosen  from  iteration  to  iteration.  In  algorithm  Ascent  strict  monotonicity  of  ||c  —  a°||  is 
guaranteed  by  choosing  a0  as  the  closest  point  to  c  in  cone(a1, . . . ,  an ).  In  algorithm  Close 
strict  monotonicity  is  maintained  by  first  checking  to  determine  if  a  there  exists  a  simple 
pivot  that  will  yield  a  new  value  a0  which  is  closer  to  c  on  the  line  segment  connecting  o° 
and  c  at  the  next  iteration.  Only  if  such  a  pivot  does  not  exist  is  aP  chosen  as  the  closest 
point  to  c  in  cone(a}, . . .,  an).  Computationally,  this  latter  option  is  far  more  attractive. 

Theorem  2  Algorithm  Close  terminates  after  a  finite  number  of  iterations  with  a  direction 
of  ascent  for  (P)  or  a  proof  that  no  such  direction  exists. 
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Proof ,  For  notational  convenience  we  let  aP  be  the  new  value  of  a0  determined  in  step  3 
of  the  algorithm  and  let  a1, . . . ,  a"  be  the  new  values  of  a1, . . . ,  a"  determined  in  step  5  of 
the  algorithm. 

We  begin  by  showing  that  after  each  iteration  of  the  algorithm  aP  6  conefa1 , . . .  ,5”). 
Initially,  a0  6  cone(a1, . ..,  an)  by  choice.  If  |/i|  >  1  in  step  3  then  aP  6  conefa1, . . . ,  a”) 
using  the  same  arguments  as  those  found  in  the  proof  of  Theorem  1.  If  \K\  —  1  then  since 
aP  €  cone(a1 , . . . ,  an)  and  7 kaP  =  0  it  follows  that  aP  can  be  written  as  a  non-negative 
combination  of  the  set  of  vectors  {a!  :  i  €  iff}.  Since  {a1  :  i  E  I§}  C  {a1, . .  .  ,a”}  by  the 
operation  of  the  algorithm  in  step  5,  a0  6  cone(a i1, . .  .,0"). 

Next,  we  show  that  when  the  algorithm  terminates  in  step  3  no  ascent  direction  exists 
and  when  it  terminates  in  step  4  an  ascent  direction  has  been  found.  Using  the  same 
arguments  as  in  the  proof  of  Theorem  1  all  constraints  a'x  <  b ,•  encountered  in  the  course 
of  the  algorithm  satisfy  a'x  =  6,-  and  if  the  algorithm  terminates  in  step  3  with  \K \  >  1  then 
no  ascent  direction  exists.  Since  a0  S  cone(a1 , . . . ,  an)  at  each  iteration  it  follows  by  the 

definition  of  a  in  step  2  that  if  d  >  1  then  c  €  cone^a1,. . . ,  a").  Thus,  c  =  u>i a1  -| - 1- u>nan 

for  some  collection  of  >  0  and  so  by  Farkas’  lemma  no  ascent  direction  in  (P)  exists 
at  x  upon  termination  in  step  3.  If  |/i|  >  1  then  using  the  same  arguments  as  in  the 
proof  of  Theorem  1  it  follows  that  if  the  algorithm  terminates  in  step  4  then  d  is  an  ascent 
direction.  Thus,  consider  the  case  |/if|  =  1.  By  the  choice  of  d  and  d  it  is  true  that 
daP  =  d{aP  +  d(c  —  a0))  =  0  and  d{aP  +  (d  +  s)(c  —  a0))  >  0  for  any  e  >  0.  Further,  since  the 
algorithm  terminates  in  step  2  if  d  >  1  it  must  be  that  d  <  1,  and  since  o°  =  a0  +  d(c  —  a °) 
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it  follows  that  c  —  a0  =  (1  —  a)  x(c  -  a0).  Thus, 

d(a°  +  (a  +  e)(c  -  a0))  >  0 
4=>-  d(a°  +  a(c  —  a0))  +  ed(c  —  a °)  >  0 
4=7  ed(c  —  a °)  >  0 
4==7  (1  —  a.)~lsd(c  -  a0)  >  0 

4=7  dc  >  daP  =  0 

which  implies  that  d  is  a  direction  of  increasing  objective  function  value.  When  the  algo¬ 
rithm  terminates  in  step  4  the  value  9  returned  by  the  oracle  for  solving  problem  STEP  is 
nonzero,  implying  d  is  a  feasible  direction  of  movement  for  (P)  at  x  and  therefore  a  true 
direction  of  ascent. 

We  continue  by  demonstrating  that  a  vector  as  with  the  properties  assumed  in  step  5 
exists.  If  | K |  >  1  in  step  3  then  an  as  with  the  desired  properties  can  be  shown  to  exist 
using  the  same  arguments  as  those  found  in  the  proof  of  Theorem  1,  so  consider  the  case 
| A' |  =  1.  Step  5  of  the  algorithm  is  reached  only  if  a  value  of  6  =  0  is  returned  by  the 
oracle  for  solving  problem  STEP,  and  by  the  definition  of  problem  STEP  it  follows  that 
dar  >  0.  As  the  set  of  points  {a1, . . . ,  a"}  —  {a5}  with  s  the  unique  element  in  K  are  linearly 
independent  and  satisfy  da1  ~  0,  it  follows  that  the  points  {a1,.  ..,an}  U  {cT}  -  {a*}  are 
linearly  independent  as  required. 

We  conclude  by  demonstrating  that  the  algorithm  is  finite.  To  this  end  we  first  show 
that  if  \K\  =  1  at  some  iteration  then  ||c  —  a°||  strictly  decreases  in  the  following  iteration. 
To  see  this,  let  k  be  the  unique  index  in  K  and  let  a1, . .  .,a"  be  indexed  so  that  ak  —  ar 
and  a?  =  a1  for  i  >  k.  Further,  let  7%  Si,  and  K  denote  the  values  of  7%  Si,  and  K  in 
the  next  iteration  of  the  algorithm.  When  \K\  =  1  the  point  a?  is  in  the  strict  relative 
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interior  of  the  facet  cone{a} , . . . ,  ak~x ,ak+1 , . . . ,  an )  of  cone^a1, . . . ,  an )  and  is  therefore  in 
the  strict  relative  interior  of  the  facet  cone(a1,. . . ,  ak~x ,  ak+1  ,...,an)  of  cone(al , . . . ,  a"). 
Since  dar  —  7 kak  >  0  it  must  be  that  7*  =  — 7*  in  order  for  7 kal  =  0  for  all  i  G  Sk  and 
7 kak  <  0,  and  with  aP  in  the  strict  relative  interior  of  this  facet  it  follows  that  a0  +  aw  G 
cone(dx, . .  .,a")  for  any  w  satisfying  7 kw  <  0  and  a  >  0  sufficiently  small.  We  show  that 
7fc(c  — a0)  <  0  as  this  will  be  sufficient  to  prove  the  strict  decrease  claim.  Note  that  7*  must 
satisfy  7 k(c  -  a °)  >  0,  for  if  not  then  since  \K\  =  1,  7 k(aP  +  a(c  -  a0))  <  0  for  all  71, ... ,  7" 
for  some  a  >  d,  which  contradicts  the  choice  of  a.  Further,  with  \K\  =  1  by  assumption, 
c  —  aP  —  c  —  ( a 0  +  a(c  —  a0))  =  (1  -  a)(c  —  a0).  Thus,  7 k(c  —  aP)  =  7fc(l  —  d)(c  —  a0)  = 
— 7fc(l  —  a)(c  —  a°)  <  0,  implying  that  aP  +  a(c  —  aP)  G  cone(ax , . . . ,  a”)  for  some  sufficiently 
small  a  >  0.  If  \K\  —  1  at  the  following  iteration  then  the  new  value  of  aP  is  aP  +  a(c  —  aP ) 
with  a  >  0,  while  if  |/f|  >  1  then  the  new  value  of  a0  is  defined  as  the  closest  point  to  c  in 
cone(al, . .  ,,an).  In  either  case,  since  aP  +  a(c  -  aP )  G  cone(dx, . .  ,,an)  for  some  a  >  0  it 
follows  that  ||c  —  a0 1|  strictly  decreases  at  the  following  iteration  as  claimed. 

Thus,  consider  a  sequence  of  iterations  in  which  \K\  =  1  and  let  a  be  the  initial  value 
of  aP  in  this  sequence  of  iterations.  At  each  iteration  aP  lies  on  the  line  segment  connecting 
a  and  c,  is  strictly  closer  to  c  than  in  the  previous  iteration,  and  satisfies  7 aP  =  0  for  some 
new  constraint  7a  =  0  uniquely  defined  by  n  -  1  points  a!  that  are  rows  of  A.  As  there  are 
a  finite  number  of  constraints  7a  =  0  defined  in  this  way,  such  a  sequence  of  iterations  must 
terminate  either  by  finding  a  direction  of  ascent,  proving  that  no  such  direction  exists,  or 
by  entering  an  iteration  in  which  \K\  >  1.  In  an  iteration  in  which  |A^|  >  1,  a0  is  redefined 
so  that  ||c  —  a°||  <  ||c  —  a||  for  all  a  G  cone(o1,. .  .,an)  so  that  ||c  —  a°||  does  not  increase. 
Further,  in  the  following  iteration  ||c-  o°jj  strictly  decreases  using  arguments  from  the  proof 
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of  Theorem  1  in  the  case  \K |  >  1  and  arguments  similar  to  those  made  above  in  the  case 
\K\  =  1.  Since  ||c  —  a°||  never  increases  it  is  never  possible  for  a0  to  again  be  contained  in 
cone(a 1,. .  .,a").  As  there  are  a  finite  number  of  cones  defined  by  subsets  of  n  rows  of  A, 
it  follows  that  the  algorithm  is  finite.  □ 

It  is  of  interest  to  compare  the  computational  work  required  at  each  iteration  of  algo¬ 
rithm  Close  and  algorithm  Steep.  When  \K\  =  1  in  algorithm  Close  it  can  be  seen  that 
the  algorithms  are  all  but  identical  except  in  step  2,  and  the  computational  burden  in  both 
algorithms  lies  almost  entirely  in  step  1.  Indeed,  for  all  practical  purposes  the  algorithms 
are  identical  in  the  amount  of  work  required  per  iteration  if  \K\  —  1.  As  an  aside,  one 
intuitively  pleasing  aspect  of  closest  ascent  is  that  it  provides  an  easily  calculated  measure 
of  progress  toward  resolution  of  degeneracy,  namely,  the  decreasing  values  of  ||c  — a°||,  which 
steepest  edge  does  not. 

The  question  remains  as  to  the  likelihood  that  \K\  =  1  in  algorithm  Close.  From  the 
definition  of  algorithm  Close  it  can  be  seen  that  at  any  iteration  of  the  algorithm  the 
probability  that  \K\  ^  1  is  the  probability  that  when  the  ray  a0  +  a(c  -  a0)  first  intersects 
one  of  n  planes  7 ka  <  0  it  simultaneously  intersects  at  least  one  other  plane.  Over  the 
course  of  the  algorithm,  the  probability  of  finding  \K\  ^  1  at  some  iteration  is  bounded 
above  by  the  probability  that  the  line  segment  connecting  c  and  the  initial  a0  chosen  in 
step  0  never  intersects  more  than  one  of  a  finite  number  of  planes  at  a  time.  The  finiteness  of 
the  number  of  planes  under  consideration  follows  from  the  fact  that  the  planes  encountered 
in  algorithm  Close  are  defined  so  as  to  contain  the  origin  and  n  —  1  linearly  independent 
rows  of  A.  As  the  following  theorem  formalizes,  the  flexibility  in  the  initial  choice  of  a0 
guarantees  that  with  probability  1  each  iteration  of  the  algorithm  will  encounter  \K\  =  1 
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independent  of  the  given  linear  program. 


Theorem  3  Let  7 =  0,  i  =  1  be  a  finite  collection  of  planes  in  IR",  let  c  be 

a  distinguished  point,  and  let  a0  be  a  point  chosen  at  random  from  any  full  dimensional 
ball  B  C  IR".  The  probability  that  any  point  a  on  the  line  segment  connecting  a 0  and  c 
simultaneously  satisfies  two  or  more  planes  at  equality  is  0. 

Proof  .  Let  Uij  denote  the  set  of  points  determined  by  the  intersection  of  the  planes 
Ya  =  0  and  7 ja  =  0  (including  the  possibility  of  0)  and  let  U  =  (J  Uij.  If  c  G  Uij  or  Uij  =  0 
let  Vij  =  Uij  and  if  c  ^  Uij  let  V, 3  denote  the  unique  n  —  1  dimensional  affine  set  of  points 
containing  Uij  and  c.  Let  V  =  (J  VtJ .  If  the  line  segment  connecting  a0  and  c  satisfies 
two  or  more  planes  at  equality  at  some  point  a  then  since  a  G  Vij  for  some  Vij  G  V  and 
c  G  Vij  it  must  be  that  a0  G  V^.  Since  V  represents  the  set  of  points  in  a  finite  collection  of 
affine  sets  of  dimension  n  —  1  or  less,  the  probability  that  a  point  chosen  at  random  in  any 
n  dimensional  ball  B  will  lie  in  V  fl  B  is  therefore  0,  completing  the  proof.  □ 

It  cannot  be  overemphasized  that  the  probability  in  the  previous  theorem  is  very  different 
from  most  other  probabilities  arising  in  the  discussion  of  degeneracy.  For  example,  the 
probability  of  degeneracy  occurring  in  a  linear  program  is  0  in  most  natural  probabilistic 
models  for  generating  a  random  linear  program,  yet  degeneracy  is  pervasive  in  most  real 
linear  programs.  The  probability  calculated  in  the  previous  theorem,  however,  hypothesizes 
a  given  linear  program  and  demonstrates  that  the  freedom  in  choosing  the  initial  value  a0 
is  what  leads  to  the  desired  probability  of  0. 

The  implication  of  the  previous  theorem  is  that  in  practice  the  need  to  find  a0  closest  to  c 
in  cone(al , . . . ,  a")  should  occur  very  rarely,  with  a  finite  probability  arising  due  to  numerical 
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issues  associated  with  finite  precision  arithmetic.  In  these  instances,  rather  than  find  such 
a  point  a 0  it  is  preferable  to  find  a  new  point  a 0  closer  to  c  through  local  adjustments  to 
a0.  Properly  developed,  such  an  algorithm  maintains  finiteness  while  allowing  for  the  extra 
freedom  to  locally  adjust  a0. 

Magnanti  and  Orlin  (1988)  observe  that  in  recent  probabilistic  studies  of  objective  func¬ 
tion  parametric  linear  programming  algorithms  “the  probability  distribution  of  a  parametric 
linear  program  is  chosen  so  that  the  problem  almost  always  satisfies  a  condition  [that  en¬ 
sures  finite  convergence  of]  the  parametric  algorithm. .  .even  if  it  is  degenerate.”  From  their 
work  it  appears  that  a  stronger  claim  similar  to  the  result  found  in  Theorem  3  can  be  made. 
When  solving  a  linear  program  using  a  parametric  programming  algorithm  there  is  usually 
some  flexibility  in  the  choice  of  initial  objective  function.  Choosing  this  objective  function 
randomly  from  the  set  of  potential  choices  should  ensure  that  degeneracy  is  not  encountered 
in  the  parametric  algorithm  independent  of  the  underlying  linear  program. 

6  Conclusions 

Two  procedures  have  been  presented  for  resolving  degeneracy  in  linear  programs.  The 
steepest  ascent  procedure,  while  not  computationally  attractive,  shed  some  light  on  the 
practical  efficiency  of  the  steepest  edge  procedure  commonly  used  in  practice  to  solve  large 
and  highly  degenerate  linear  programs.  Ideas  found  in  the  steepest  ascent  procedure  were 
then  used  to  develop  a  closest  ascent  procedure  that  with  probability  1  had  nearly  identical 
computational  requirements  per  iteration  as  steepest  edge  but  was  guaranteed  to  finitely 
terminate.  All  of  the  procedures  discussed  had  the  unique  advantage  that  the  pivot  element 
is  chosen  without  knowledge  of  the  set  of  all  active  constraints  at  a  point  of  degeneracy. 
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A  number  of  interesting  questions  remain.  While  the  author  suspects  that  there  exist 
examples  in  which  steepest  edge  leads  to  an  infinite  number  of  pivots,  the  question  of 
whether  or  not  steepest  edge  is  a  degeneracy  resolution  procedure  remains  unresolved. 
The  relative  number  of  iterations  required  in  practice  by  steepest  edge,  steepest  ascent,  and 
closest  ascent  also  remains  an  interesting  question.  Steepest  ascent,  while  probably  requiring 
the  fewest  iterations,  is  likely  to  be  too  expensive  per  iteration  to  warrant  consideration  in 
linear  programming  codes.  However,  the  number  of  iterations  required  by  closest  ascent 
should  be  similar  to  the  number  required  by  steepest  edge,  in  which  case  closest  ascent 
should  prove  to  be  an  effective  practical  procedure  for  degeneracy  resolution. 
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